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Pattern  Formation 


Many  systems  have  been  studied  as  models  for  spatial  pattern  formation  in  biology,  chemistry  and 
physics.  The  types  of  systems  we  are  interested  are  those  that  consist  of  a  large  number  of  ‘cells’ 
in  a  regular  array.  The  state  of  each  cell  represents  an  important  quantity,  such  as  a  chemical 
concentration  or  a  magnetic  spin  direction,  and  changes  in  time  according  to  a  rule  which  depends 
on  the  state  of  the  cell’s  neighbors.  The  primary  example  is  the  reaction-diffusion  system  used  to 
explain  symmetry  breaking  in  biological  growth.  Other  examples  include  cellular  automata  models 
for  Ising  spin  glass  systems  and  spatially  coupled  differential  equations  and  maps  for  binary  alloys. 
These  systems  can  produce  some  very  qualitatively  similar  output,  suggesting  a  common  underlying 
mechanism.  We  have  used  the  CNN  paradigm,  which  has  a  more  general  and  accessible  parameter 
space,  to  demonstrate  principles  which  lead  to  such  patterns  and  to  study  what  types  of  patterns 
are  possible. 

A  theoretical  generalization  of  these  paradigms  was  developed  for  the  CNN  which  can  be  used 
to  predict  the  pattern  formation  properties  of  given  templates,  based  on  the  Turing  instability 
approach  and  the  synergetics  paradigm.  It  is  our  opinion  that  the  Cellular  Neural  Network  model 
provides  a  superior  method  to  control  the  critical  instabilities  needed  for  pattern  formation  without 
obfuscating  ])arameterizations,  complex  noiiliiiearities.  or  high-order  cell  states,  which  allows  a 
general  and  convenient  investigation  of  the  essence  of  the  j)attern  formation  properties  of  these 
systems. 

Turing  Instability 

We  have  found  that  the  Ollular  Neural  Network  can  produce  patterns  similar  to  those  found  in 
Ising  spin  glass  systems,  discrete  l)istal)le  systems,  and  the  reaction-diffusion  system.  In  our  study, 
we  did  not  restrict  the  coupling  between  cells  to  be  with  immediate  neighbors  only  or  to  have  a 
S])ecial  diffusive  form.  When  larger  neighborhoods  and  generalized  diffusion  coupling  is  allowed, 
it  was  found  that  some  new  and  unique  j)atterns  can  be  formed  that  do  not  fit  the  standard 
ferro-antiferromagnetic  paradigms. 

We  found  that  the  first  order  ('NN  can  exhil)it  the  critical  spatial  frequency  instability  around 
the  equilibrium  required  for  Turing  pattern  formation.  This  is  analogous  to  a  one-morphogen  or 
single-layer  reaction  diffusion  system,  which  can  exhibit  the  critical  instability  when  negative  or 
larger  diffusion  matrices  are  allowed.  The  (JNN  presents  a  natural  way  to  study  generalizations  of 
diffusion  and  what  types  of  instabilities  are  possible.  In  addition,  the  simple  saturation  nonlinearity 
of  the  CNN  locally  enforces  a  frequency-phase  separation  which  is  found  in  many  real  pattern 
forming  systems. 

Our  research  has  led  us  to  identify  the  following  three  epochs  of  behavior  in  the  time  formation 
of  patterns  which  are  generated  by  random  perturbations  about  the  unstable  equilibrium. 

1.  Linear  System  leading  to  noise  shaping.  If  the  system  is  halted  at  the  moment  when  the 
first  cell  of  the  array  enters  the  saturation  region,  the  linear  system  theory  will  be  exact. 

The  states  at  this  time  will  be  the  result  of  a  linear  filtering  operation  which  can  be  found 
in  terms  of  the  template  weights.  Such  a  filtering  operation  can  be  understood  to  enhance 
certain  spatial  frequencies  while  suppressing  others. 

2.  Separation  of  Spatial  Frequencies  and  Phases  leading  to  meta-stability.  In  regions  where  a 
significant  number  of  cells  have  reached  saturation,  the  linear  analysis  does  not  even  hold 
approximately.  At  this  point,  arbitrary  combinations  of  the  unstable  modes  can  no  longer  be 
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maintained.  The  states  begin  to  separate  themselves  into  regions  with  motifs  that  are  at  least 
locally  stable.  These  regions  grow  until  their  boundaries  meet  with  other  regions  of  different 
motif.  At  this  point  the  system  is  considered  to  be  meta-stable  in  the  sense  that  most  cells 
are  unchanging  in  time  (the  interior  cells  of  the  regions)  and  only  the  boundaries  between 
regions  are  moving. 

3.  Boundary  Negotiation  leading  to  stability.  Even  though  the  array  has  separated  into  regions 
that  are  locally  stable,  the  boundary  between  them  may  prove  to  be  unstable.  Thus  begins 
a  long  process  where  most  of  the  cells  in  the  array  are  not  changing  in  time,  but  slowly  the 
boundaries  are  negotiated.  In  some  cases  there  may  not  be  any  acceptable  boundary,  and 
eventually  one  motif  must  cover  the  whole  array.  The  whole  process  may  take  many  order  of 
magnitudes  longer  than  the  first  two  steps; 

The  simple  form  of  the  CNN  nonlinearity  allowed  us  to  observe  a  more  defined  distinction 
between  the  growth  due  to  linear  instability  in  the  linear  region  and  phase/frequency  separation 
and  boundary  negotiation  in  the  saturation  region. 

We  have  also  investigated  the  patterns  formed  from  site  defects  and  irregular  array  boundaries. 

Synergetics 

The  study  of  the  s])ontaneous  self-organizing  behavior  of  collections  of  locally  coupled  systems  is 
known  as  synergetics.  The  name  comes  from  the  idea  that  cooperation  between  simple  systems  can 
produce  a  large-scale  order.  Synergetic  mechanisms  have  been  found  to  be  useful  for  explaining 
phenomena  in  i)hysics,  chemistry,  and  biology.  In  addition,  an  understanding  of  those  mechanisms 
has  been  a|)plied  to  problems  in  computer  recognition  of  human  faces  under  the  philosophy  ^'Pattern 
Recognition  as  Pattern  FormatioiP’. 

The  ])rinc.iple  ideas  from  synergetics,  as  ])romoted  by  II.  Haken,  are  as  follows:  First,  a  syn¬ 
ergistic  system,  when  linearized  about  an  unstable  e(|uilibrium,  can  be  characterized  in  terms  of 
stal)le  and  unstable  modes.  Small  perturbations  about  the  e(|uilibrium  will  excite  these  modes. 
The  unstable  ones  will  begin  growing  wliile  the  stal)le  ones  diminish.  The  stal)le  modes  can  be 
found  to  be  'slaved’  to  the  unstable  ones.  That  is,  the  dynamic,  trajectory  of  the  stable  modes 
will  not  de])end  on  the  perturbation  in  the  stable  direction  but  can  be  written  solely  in  terms  of 
the  unstable  modes.  Second,  the  nonlinearity  will  eventually  promote  a  competition  among  the 
unstable  modes  for  dominance.  The  relative  strength  of  the  unstable  modes  are  known  as  order 
parameters  since  they  solely  determine  the  type  and  amount  of  organization  in  the  equilibrium. 

We  have  shown  that  the  nonlinear  array  dynamics  of  the  simplest  (Jellular  Neural  Network 
(CNN)  exhibits  the  basic  phenomena  of  ‘synergetics’.  We  have  investigated  how  the  computation 
performed  by  physical  synergetic  systems  could  be  eiiudated  in  a  natural  way  on  the  CNN  to  perform 
useful  image  processing  tasks.  Namely,  the  physical  phenomena  can  be  interpreted  in  terms  of  linear 
spatial  filtering  and  morphological  constraints.  The  classical  CNN  ‘averaging’  template  has  been 
interpreted  in  this  light  and,  informed  by  the  synergetics  interpretation,  improved  for  a  particular 
example.  We  have  also  investigated  applications  to  fingerprint  enhancement  and  strong  feature 
detection. 


Spatial  Disorder 

Cellular  Automata 

The  cellular  automata  are  a  class  of  dynamical  systems  which  are  discrete  in  state,  space,  and  time. 
The  states  of  a  cellular  automaton  evolve  synchronously  according  to  a  state  transition  mile  which 
operates  within  a  local  neighborhood  of  some  radius  of  the  state  being  updated,  typically  in  a  time 
and  space-invariant  manner.  The  order  of  the  (JA  is  the  number  of  previous  times  the  states  are 
recpiired  to  computed  the  next  state. 

We  have  given  a  general  constructive  proof  that  the  CNN  Universal  Machine  architecture  can 
directly  implement  arbitrary  first-order  cellular  automata.  Past  approaches  have  used  either  dis¬ 
crete  time  operation  and  complex  nonlinearities,  multiple  layers,  or  time  varying  templates.  By 
using  the  CNN  Universal  Machine  architecture,  we  show  that  these  complications  are  unnecessary. 
In  addition,  we  have  also  shown  that  a  ])articular  and  useful  class  of  reversible  second-order  CA 
can  be  implemented  on  the  (JNNUM. 

Random  Number  Generation 

High-speed  high-quality  reproducible  random  number  generation  is  a  significant  problem,  with 
many  applications  ranging  from  simulation  of  physical  systems  to  cryptography.  We  have  demon¬ 
strated  that  the  (,'NN  Universal  Machine  (or  the  Discrete-time  (!NN)  is  capable  of  producing  a 
two-dimensional  i)seudo-random  bit  stream  at  higli  speeds  by  means  of  cellular  automata  (CA)  We 
investigated  methods  for  random  number  generation  on  the  (bNNUM  with  two  reasons  in  mind: 
First  as  a  native  source  of  good  random  numbers  for  use  in  (’NNUM  algorithms  for  such  areas  as 
image  processing  and  modeling  physical  systems,  and  second  as  a  liasis  for  implementing  excep¬ 
tional,  and  possibly  novel,  cryptographic  algorithms.  However,  there  are  many  ])otential  uses  for  a 
CNN  based  pseudo-random  number  generator,  for  instance,  random  location  selection,  annealing 
algorithms,  image  dithering  and  quantization,  stochastic  resonance,  Caussian  noise  generation  and 
s])read  spectrum  communications. 

The  first  random  number  generation  schemes  which  we  develoi)ed  for  the  (.'NNUM  were  based 
on  irreversible  two-dimensional  CA  rules  and  were  analyzed  by  a  battery  of  statistical  tests  as  well 
as  mean-field  theory.  Second,  a  special  class  of  reversible  second-order  (’A,  which  are  also  easily 
implemented  on  a  ('NNUM,  were  considered  for  random  number  generation  and  were  shown  to 
have  some  desirable  properties.  These  second-order  reversible  CA  exhibit  gas-like  behavior. 

The  randomness  of  CA  have  been  investigated  in  other  places,  but  have  only  been  well-analyzed 
for  a  certain  one-dimensional  rule.  Some  two-dimensional  CA  have  been  noted  to  be  random,  but 
have  not  been  sufficiently  analyzed.  Our  findings  indicate  that  the  two-dimensional  ('A  show  great 
promise  as  random  number  generators. 

Cryptography 

Finally,  as  an  example  application  for  random  number  generation  on  the  CNNUM,  some  crypto¬ 
graphic.  schemes  were  proposed.  Due  to  its  massive  parallelism,  the  CNNUM  offers  the  possibility 
of  high  speed  cryptography  while  offering  very  large  (>  1000  bits)  key  space. 

Perhaps  the  simplest  use  of  CNN-l)ased  ])seudo-random  bit  generation  for  cryptographic  pur¬ 
poses  is  in  a  key  stream  cipher.  In  this  scheme,  a  stream  of  ‘random’  bits  from  a  key  stream 
generator  is  used  to  encode  the  plain-text  data  stream  bit-by-bit  by  applying  an  XOR  operation 
(which  is  invertible)  between  them.  At  the  receiver,  the  same  sequence  of  bits  produced  by  an 


identical  key-stream  generator  is  then  used  to  decode  the  plain-text.  Security  is  provided  by  the 
use  of  a  key,  which  is  a  shared  secret,  to  somehow  affect  the  bit-stream  produced. 

Another  possibility  is  the  block  cipher,  which  transforms  blocks  of  the  plain-text  during  encryp¬ 
tion.  This  type  of  approach  has  many  advantages  including  the  facts  that  the  sender  can  easily  mix 
random  data  into  the  plain-text  stream  so  that  sending  the  same  message  will  result  in  a  different 
cipher-text  and  the  key  or  key  stream  is  not  easily  found  through  a  plain-text  attack. 

In  our  proposed  block  encryption  scheme,  under  investigation,  we  use  the  reversible  CA  as  the 
basis  for  the  algorithm.  The  initial  states  — 1)  and  .s(0)  are  set  to  be  an  arbitrary  random  image 
and  the  data  image  respectively.  Then,  the  CA  is  run  for  a  previously  defined  number  of  steps,  A, 
and  s{N  -  1)  and  s{N)  are  transmitted.  The  trick  is  to  introduce  some  secret  (shared)  information 
into  the  CA  dynamics.  This  could  be  many  different  things  including  switching  between  CA  rules 
according  to  a  secret  ordering,  flipping  state  bits  in  certain  secret  places  and  times,  or  specifying 
boundary  value  information.  The  receiver  can  then  run  the  CA  system  backwards  to  recover  the 
initial  data. 
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